Hierarchical transformation invariant clustering

We present a novel generative model for transformation invariant data clustering, with the emphasis on video clustering and summarization. We model the data as the mixture of transformed exemplars. We include in the model the most common and realistic transformations in video data: camera motion, cropping, change in scale and light. The learning procedure automatically clusters video frames into video scenes and objects. The learning algorithm is based on a hierarchical, on-line EM algorithm. Fast Fourier transform is used for rapid inference and learning. We use the model to: 1. perform video clustering by grouping similar (up to translation, scale and lighting) video frames into clusters; 2. robustly stabilize video frames by compensating for translation, scale and lighting changes. We believe that video scene modelling of this kind is essential to bridge the semantic gap in video understanding. We illustrate this with several excellent results, both in terms of speed and accuracy.